1. Field of the Invention
This invention relates generally to the field of devices to enable visually impaired people to determine information about objects in their vicinity.
2. Description of the Prior Art
In general, devices for capturing and measuring distances, temperatures, and optically spectral differences are known in the prior art. These devices are commonly in the form of 3D-sensors, rangefinders, cameras, and temperature sensors. Common methods and procedures used to make such measurements include measurement of the running time, photonic mix devices, stereopsis, and triangulation procedures. The representation of these data and the communication of these data is conventionally in a two-dimensional format, such as on a visual screen, a printout, 3D-plots or other two-dimensional representations, by acoustic signals, or by a combination of the two.
Such devices do not offer meaningful information to the blind. Those devices using a visual screen or a printout are of no use to the visually impaired and those devices that use an acoustic tone or signal give the exceedingly narrow information of a warning of the presence of a tangible object. However, a problem common to all these devices is that some tangible objects are very difficult to detect and yet still pose a hazard for the visually impaired. By way of example, most devices would tend to have trouble detecting a chain link fence up close or a glass wall at night.
Visually impaired persons need more than an alert that a tangible object has been detected; they need information about the object that has been sensed.
The following 12 patents and 2 European patents are relevant to the field of the present invention.
1. U.S. Pat. No. 3,337,839 issued to Arthur Nelkin and assigned to Westinghouse Electric Corporation on Aug. 22, 1967 for “Vetrasonic Guidance Apparatus” (hereafter the “Nelkin Patent”);
2. U.S. Pat. No. 3,546,467 issued to Joel Malvern Benjamin, Jr. et al. on Dec. 8, 1970 for “Typhlocane With Range Extending Obstacle Sensing Devices” (hereafter the “'467 Benjamin Patent”);
3. U.S. Pat. No. 3,654,477 issued to J. Malvern Benjamin, Jr. and assigned to Bionic Instruments, Inc. on Apr. 4, 1972 for “Obstacle Detection System For Use By Blind Comprising Plural Ranging Channels Mounted On Spectacle Frames” (hereafter the “'477 Benjamin Patent”);
4. U.S. Pat. No. 4,310,903 issued to Leslie Kay on Jun. 12, 1982 for “Method Of And Apparatus For Providing Information As To The Existence Or Position Of Objects” (hereafter the “Kay Patent”);
5. U.S. Pat. No. 4,712,003 issued to Itsuki Ban et al. on Dec. 8, 1987 for “Blind Person Guide Device” (hereafter the “Ban Patent”);
6. U.S. Pat. No. 5,687,136 issued to Johann Borenstein and assigned to The Regents of the University of Michigan on Nov. 11, 1997 for “User-Driven Active Guidance System” (hereafter the “Borenstein Patent”);
7. U.S. Pat. No. 5,717,392 issued to Marty Eldridge on Feb. 10, 1998 for “Position-Responsive, Hierarchically-Selectable Information Presentation System And Control Program” (hereafter the “Eldridge Patent”);
8. U.S. Pat. No. 5,807,111 issued to Jens Schrader on Sep. 15, 1998 for “Orientation Aid” (hereafter the “Schrader Patent”);
9. U.S. Pat. No. 5,973,618 issued to Christ G. Ellis on Oct. 26, 1999 for “Intelligent Walking Stick” (hereafter the “Ellis Patent”);
10. U.S. Pat. No. 6,198,395 issued to Gary E. Sussman on Mar. 6, 2001 for “Sensor For Sight Impaired Individuals” (hereafter the “Sussman Patent”);
11. U.S. Pat. No. 6,298,010 issued to Maria Ritz et al. on Oct. 2, 2001 for “Orientation Aid For The Blind And The Visually Disabled” (hereafter the “Ritz Patent”);
12. U.S. Pat. No. 6,486,784 issued to Fabien Beckers on Nov. 26, 2002 for “Process And System Enabling The Blind Or Partially Sighted To Find Their Bearings And Their Way In An Unknown Environment” (hereafter the “Beckers Patent”);
13. European Patent No. EP0669119 issued to Fischer UWE on Aug. 30, 1995 for “Guiding Device” (hereafter the “Fischer European Patent”);
14. European Patent No. EP0749744 issued to Guenther Detlef on Dec. 27, 1996 for “Guiding Apparatus For The Blind” (hereafter the “Detlef European Patent”).
The Nelkin Patent which issued in 1967 discloses the broad general concept of having a device which scans the area in front of the person and provides a tactile stimulation in different areas of the device so that the person after he is trained in what the tactile stimulations mean will be able to determine recognition of what the device in front of the person is.
Referring to the Nelkin Patent, the preferred embodiment of the ultrasonic guidance apparatus as shown in FIGS. 1 and 4 appears to show that the device is in the shape of a vest or a belt worn around the person. The display is strapped onto the body of the user both front and back. Additionally, the Nelkin apparatus is large and it appears that both the front piece and the back piece each cover an area that is nearly the entire width of a person's torso. The device is large, bulky and not easy to handle.
Another problem with the Nelkin Patent is the difference between its design and its practical use. The Nelkin apparatus is designed such that the indicator of the presence of an object is “an array of tactile-stimulus producers coupled to a skin area of the user's body.” (Col. 1, lines 67-68) (emphasis added). However, in referring to FIG. 1, which is an accurate representation of how a user would wear the device, the apparatus, both the front piece and the back piece, are worn on the outside of the clothes. Therefore, the coat or other garments between the device and the individual block some of the signals and reduce the quality of the signals sensed by the user. However, this may be necessary to protect the user's body from pain and bruising, as the Nelkin apparatus makes a complete scan of all nine transducers eight times a second in short range mode, six times a second in medium range mode, and four times a second in long range mode, which potentially results in the Nelkin apparatus pummeling the wearer's back either seventy-two times a second in short range mode, fifty-four times a second in medium range mode, and thirty-six times a second for the long range mode.
The Nelkin Patent discloses the broad concept of scanning the area in front of the person to provide feedback signals as to what the scan detects. However, this device is limited to ultrasonic scanning. Further, the Nelkin apparatus is merely a blunt, obstacle recognition device. The Nelkin Patent claims that when scanning about seven feet from the operator, the 3×3 grid of “transducer beams cover roughly an outline of 4 feet high by 4 feet wide” (Col. 2, lines 65-66). This indicates that each transducer is sensing an area of 16 inches×16 inches. This means that if a transducer receives feedback of anything within the 256 square inch area that it is sensing, the tactile sensing device will be triggered. This means that the Nelkin device is incapable of distinguishing small or large objects or make finer sensing, like the outline of a cup, but instead, the operator will merely get a thump on the back that an object has been sensed, not knowing if the Nelkin Patent sensed a cup, a plant, or a television. With regard to larger objects, the Nelkin Patent is incapable of distinguishing between a door, a wall, a cave, a group of people, a hedge, a fence, or an elephant. Further, where there are lots of people, the Nelkin Patent will become useless and merely pummel the operator ceaselessly without conveying any pertinent or new information to the operator. The Nelkin Patent merely gives the operator an extremely large pixellated sense of the mere presence of an object and is not capable of allowing the operator to “see” what lies ahead by offering a sharper, finer image of what is sensed.
The '467 Benjamin Patent has a cane with a sensing device to pick up if there is an obstacle in the path of the cane and provide a signal from a signaling means that pokes the operator, which is shown as 306 in FIG. 4. The Benjamin Patent is a fairly involved sensing apparatus which senses an object in the path of the cane and provides a signaling means to the person holding the cane so that the operator is poked by the signaling means to advise the person that there is an object in the path of the cane.
The '467 Benjamin Patent sensors their corresponding means to notify the operator do not give the operator a picture or a sense of what is in front of the operator. In referring to the '467 Benjamin Patent, the preferred embodiment of the invention shown in FIG. 1, appears to show that the apparatus does not have a complete line of site, but instead, has a disjointed sensing means for sensing three small ranges, the views above, at thigh level, and at ground level of the operator. Like the Nelkin Patent, the '467 Benjamin Patent merely acknowledges the presence of an obstacle in any one of those three areas to the operator. It does not convey to the operator the different details about what the object is, but merely conveys that an object is near. Additionally, the '467 Benjamin Patent senses objects by receiving its reflection; each of the three projected beams “is approximately ½ degree. Thus at 10 feet the beam is about 2 inches long and ½ inch high.” (Col. 6, lines 50-52). Therefore, the '467 Benjamin Patent only provides the operator with three very small windows of the world ahead. While the upper two sensors of the '467 Benjamin Patent detect the presence of an object, the bottom sensor is designed to detect the absence of an object. Further, '467 Benjamin Patent notifies the operator of the presence or absence of an object in a general manner. The means to notify the operator of an object sensed by the middle sensor is by means of “a tactile stimulator 305 [that] includes a poker 306, which projects through the casing of the shank.” (Col. 11, lines 43-45). For the upper sensor which detects the presence of an object and the lower sensor which detects the absence of an object, the means to notify the operator is “by an audible tone” (Col. 11, line 54). The '467 Benjamin Patent is simply a cane with using laser scanners that give feedback, regarding presence or absence of obstacles in three very specific areas, in very general terms.
The '477 Benjamin Patent is an obstacle detecting device which, also, through the unique electronics of the device, indicates the presence of an obstacle in the operators's path. However, the device is merely a warning system. The primary focus of this invention is to incorporate the obstacle detecting and warning device onto an eyeglass frame, which when worn by the operator will poke the operator behind the ear when an obstacle is detected.
The inventor of this patent is the same individual as the Mr. Benjamin from the previous patent and, again, discloses the concept of scanning the area in front of the operator to pick up signals with the purpose of this invention is to warn the operator that there is an obstacle in front of the sensors without giving any details about the obstacle. This invention has many disadvantages. The invention utilizes Fresnel lenses which are used in applications where imaging quality is not critical. Further, this device detects obstacles by measurement of the laser run time, “the time between the transmission and reception of the pulse.” (Abstract). The '477 Benjamin Patent has a very annoying means of alerting the operator to the detected presence of an obstacle. A small rod 44 protrudes from the ear piece 30 to poke the back of the ear to warn of an impending obstacle. Further, the hole 56 out of which the small rod 44 projects has a forward bearing 58 that extends slightly beyond the body of the earpiece 30. Since the ear piece must be very closely situated behind the ear so that the small rod can reach the back of the ear when actuated, the forward bearing 58 is most likely rubbing against the back of the operator's ear. If the rod 44 does not fully return to the interior of the hole 56, then the rod 44, will also be rubbing against the back of the operator's ear. Another disadvantage is that the power supply or battery for this device is not incorporated into the eyeglass frame, but instead resides in its own unit or canister 28, for which the operator must accommodate, either by wearing a jacket with a breast pocket or some other means, but having some sort of way to protect the operator from the heat from the power supply and yet allow the power supply to dissipate its heat. Further, the '477 Benjamin Patent is dangerous. As illustrated in FIG. 4, the optical system resides behind the lens of the eyeglass frame, such that all the lasers, photopickups, and etc., are entirely “accommodated in the space between the lens area and the wearer's eye.” (Abstract). As shown in FIG. 4, after accommodating all the electronics, there is very little space between the device and the eye of the wearer. Should the wearer get bumped or fall or even be jostled, the back of the device would be in various degrees of contact with the eye of the wearer and cause damage and pain. Between the rubbing and the poking on the back of the ear, dealing with a loose battery pack that inevitably heats up, and the possible loss of the eyeball, the operator may find this device very uncomfortable to use and wear.
The Kay Patent also discloses the concept of having an ability to detect objects for the visually impaired by transmitting a stress wave which is reflected back to the device to provide the information to the person about an object in the person's path. Using sound waves in the supersonic frequency range, the Kay device modulates the frequency of the sound waves and determines the range from the object by the magnitude of the frequency difference in the received signal. The Kay Patent is focused on the method of determining distance and only in very general terms discusses how the object range could be communicated to the operator by sight, sound or touch. For tactile sensing, the Kay Patent teaches a matrix of rows and columns of pins that will vibrate to indicate a sector where an object has been located. There is no evidence that the Kay Patent is capable of being a handheld or portable device.
The Ban Patent discloses a device incorporated onto a cane, which using either infra-red or sonar, sends out a signal and receives a return signal to tell a blind person as to the distance the object is from the blind person. The Ban device is capable of measuring only one point at a time, merely warns the operator of the presence of an object, and does so by vibrating the cane.
The Borenstein Patent is a user driven activation guidance system. It is a large device on a pair of wheels and rolls in front of the operator who must push it from the long handle extending from the device to the operator. The device functions as an obstacle avoidance system. It uses an array of ultrasonic sensors to detect distance and location of obstacles. The device uses a servomechanism to steer the device away from the obstacle. The operator who is pushing the device, must also merely follow the device as objects are sensed and avoided. The operator has no idea of or any information about the object around which the Borenstein device has guided him.
The Eldridge Patent is a device using GPS to tell the location and direction of the operator and can be incorporated into a vehicle, to help the operator to go from one location to another. The device uses a display to convey its information, which can be in a Braille format, but beyond location and direction of the operator, the device conveys no information about the operator's surroundings.
The Schrader Patent is an orientation aid to help the visually impaired walk in a straight line or to return to a path after avoiding an obstacle. The device, incorporated into a belt worn by the operator, signals the operator, either on the left or the right side of the operator, to correspond to the direction the operator needs to move in order to return to the former path. This device merely advises the operator of the current compass direction in comparison to a former compass direction.
The Ellis Patent is a large system to encompass an entire city or larger to communicate and control the infrastructure, e.g. communicate traffic information to the main system and smaller systems, such as in a vehicle, and control the traffic lights to stop traffic and give emergency vehicles right of way. One part of this system is a device, to be used by pedestrians, housed in a cane, walking stick, or belt that receives signals by transmitters and receivers in the walking stick and communicates with the pedestrian operator. The device includes a processor, a transmitter receiver and an outside image sensor or scanner, and a warning device. The scanner may, for example, sense the shape of a traffic signal or color of a traffic signal and provide a signal back to the person. The device used by the pedestrian also transmits information, such as to automobiles who have receivers to communicate either visually, or aurally with the driver that a pedestrian is near by, and may initiate the braking system on the car so as not to hit the pedestrian. The walking stick for pedestrians in the Ellis Patent is merely a warning device and does not convey information about the physical characteristics of the pedestrian's surroundings.
The Sussman Patent discloses the concept of providing signals which are picked up from laser transmitted signals to advise the blind person about different areas around the person. The device merely communicates distance of a detected object and communicates this information by the use of either vibrations on the skin, an audio amplifier, or tightening bands around the fingers.
The Ritz Patent is a one point distance recognition system that conveys distance to the operator. Distance is communicated to the operator by a pin that slides horizontally along the side of the device. Distance is indicated by the distance the pin slides from the zero position to anywhere along a three centimeter track. The scale of distance of the three centimeter track can be changed by the operator
The Beckers Patent has an invention which leads the operator from position to position. The system enables the operator to find his bearings in an unknown environment. Although this is a very involved technical patent with the various means to calculate distances and other information about the surroundings of the blind person, it is merely a position marker orientation system. The device does not provide any object recognition, nor does it communicate any information about the operator's surroundings.
The Fischer European Patent Application discloses a guidance device that is used for orienting a blind person within an environment, and includes a radar signal transmitter and a radar signal receiver. The device is incorporated into a helmet that transmits the signals which receives return information and requires a wire to connect the instruments in the helmet to the radar signal sender that resides in a belt worn around the waist. This invention teaches only an obstacle recognition device.
The Detlef European Patent Application discloses a guidance device used for a blind person is built in two parts. One part includes a receiver and a transmitter. The second part includes a microprocessor for processing the signals from the receiver. The second part of the guide also includes a mechanical signal generator, a current supply and a charger. The second part can be held in the blind person's breast pocket. The first part includes the current supply, batteries and charging circuit. The transmitter emits ultrasound waves which are reflected by an obstacle. The reflected signal warns the user about the presence of an obstacle.
Overall, the prior art teaches the broad concept of obstacle recognition and avoidance devices. However, these devices are unsatisfactory. The main reason is that these devices don't allow their operators to ‘see’ the objects detected. Generally, the blind and visually impaired must touch an object to ‘see’ it; in doing so, the blind can feel the shapes, contours, textures, and temperatures of the objects with their fingertips. While the prior art alerts the operator that an object is present and needs to be avoided, that is all the information conveyed to the operator. The devices can't convey to the operator that the object detected is a mailbox or child. These devices can't direct the blind to the green door as opposed to the red door, nor can it alert the blind that there is a fire behind the door they are approaching. These devices can only operate in the open air and cannot aid the visually impaired under water.